3D time of flight active reflecting sensing systems and methods

ABSTRACT

The system and method provide for identification of dynamic objects in an enclosed space and the presence of a component in a primary location. The system uses an active electro-optical 3D sensor, such as a three-dimensional time of flight camera, to identify the presence or absence of a reflected pulse, to determine, for example, proper placement of a seat belt, or a change in characteristics of a reflected pulse to determine a change in location, and thus possible movement, of a living creature in a vehicle, for example.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority to and incorporates by reference U.S.Provisional Patent Application 62/431,185 filed on Dec. 7, 2016, andentitled “3D Time of Flight Active Reflecting Sensing Systems andMethods.”

TECHNICAL FIELD

This disclosure relates to monitoring of vehicle non-line of sightregions with active optical sensors and optically reflectiveintegration, and more particularly to using a Time of Flight ReflectingSensing System to detect changes in reflected images to detect thepresence of a moving object.

BACKGROUND

Systems to detect the presence of a child in a child seat have beendeveloped and tested where the sensing systems are integrated directlyinto the child seat structure, as indicated in recent NHTSA report (DOTHS 812 187, July 2015). These systems are based on detection of thechild seat through several sensing mechanisms, including the measurementof pressure, force, latching clip detection, etc. Such systems aredesigned to include electronics within or on the seat for detection andcommunication to/from the vehicle and/or to/from cell phones. Thesystems contain microprocessors that take the sensor and vehicleinformation as inputs, and monitor conditions where the child may beleft behind, and a range of potential countermeasures can be taken,including initiating a warning through the vehicle systems (e.g. hornalarm); modifying the vehicle cockpit (e.g., reducing/increasing cabintemperature); and initiating a warning through telematics (e.g., sendinga warning to the parent/driver).

Additionally, detection of child seat occupancy has been studied using atwo-dimensional (2-D) camera, a three-dimensional (3-D) camera and otheractive electromagnetic methods such as ultrasonic, radar, and acoustics.These systems have shown the potential to detect the child seats andclassify them as occupied or un-occupied. Another potential method isbased on the use of seat weight detection systems, possibly includingbelt tension sensor(s). All of the above methods can be prone toincorrect classifications, due to clothing, due care, blockingobstructions, lack of motion by the occupant, etc. Moreover, many ofthese systems depend on sensing an expected location of an occupant, andcannot account for an occupant being separated from the sensed location.

SUMMARY

Accordingly, the present invention is directed to 3D time of flightactive reflecting sensing systems and methods that address one or moreof the problems due to limitations and disadvantages of the related art.

An advantage of the present invention is to provide a system fordetecting dynamic objects in an enclosed space with an activeelectro-optical 3D sensor. For spaces out of the field of view of thesensor, a reflective surface capable of reflecting a wavelengthcorresponding to the active electro-optical 3D sensor is utilized. Thereflective surface is in line of sight with the active electro-optical3D sensor and is in line of sight of at least one volume of the enclosedspace that is not in line of sight with the active electro-optical 3Dsensor. If characteristics of light reflected by the reflective surfaceto the active electro-optical 3D sensor at a first time differ fromcharacteristics of light reflected by the reflective surface to theactive electro-optical 3D sensor at a second time, the sensor indicatesa dynamic object is present in the enclosed space.

In another aspect of the present invention, further embodiment ofdetecting proper positioning of a device includes an activeelectro-optical 3D sensor and a reflective surface capable of reflectinga wavelength corresponding to the active electro-optical 3D sensor. Thereflective surface is on a surface of a component such that, if thecomponent is in a primary position, the reflective surface is in line ofsight with the active electro-optical 3D sensor and out of line of sightwith the active electro-optical 3D sensor if the component, or a portionof the component, is displaced from the primary position. Ifcharacteristics of light reflected by the reflective surface to theactive electro-optical 3D sensor at a first time differ fromcharacteristics of light known to indicate the component is in theprimary position, the active electro-optical 3D sensor indicatesdisplacement of the component.

An associated method includes detecting an object in a vehicle cabin,transmitting a pulse from a three-dimensional time of flight camera, anddetecting a presence or absence of a returned pulse reflected to thethree-dimensional time of flight camera. The method further includesmeasuring changes in characteristics of the returned pulse to determinechange in objects within the enclosed space.

Further embodiments, features, and advantages of the sensor systemdisclosed herein, as well as the structure and operation of the variousembodiments of the system and method, are described in detail below withreference to the accompanying drawings.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory only,and are not restrictive of the invention as claimed.

DESCRIPTION OF DRAWINGS

The accompanying figures, which are incorporated herein and form part ofthe specification, illustrate a 3-D time of flight (TOF) activereflecting sensing system and methods. Together with the description,the figures further serve to explain the principles of the 3-D time offlight (TOF) active reflecting sensing system and methods describedherein and thereby enable a person skilled in the pertinent art to makeand use the same.

FIG. 1 illustrates a concept of detecting movement by monitoringsecondary reflection in a vehicle cabin.

FIG. 2 illustrates monitoring vehicle occupants by location and properseat belt use through a reflective pattern on a seat belt.

FIG. 3A shows an example of depth images of a vehicle footwell monitoredthrough a fixed mirror according to principles of the presentdisclosure.

FIG. 3B shows an example of intensity images of a vehicle footwellmonitored through a fixed mirror according to principles of the presentdisclosure.

FIG. 4A shows an example of intensity images from a side view of avehicle footwell monitored through a fixed mirror at time T1 accordingto principles of the present disclosure.

FIG. 4B shows an example of intensity images of a vehicle footwell asmonitored from a side view through a fixed mirror at time T2 accordingto principles of the present disclosure.

FIG. 4C shows top view intensity images of a vehicle footwell monitoredthrough a fixed mirror according to principles of the presentdisclosure.

FIG. 4D shows top view intensity images of a vehicle footwell monitoredthrough a fixed mirror according to principles of the presentdisclosure.

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION

The details of one or more embodiments of the disclosure are set forthin the accompanying drawings and the description below. Other features,objects, and advantages of the disclosure will be apparent from thedescription and drawings, and from the claims. It will be apparent tothose skilled in the art that various modifications and variations canbe made in the present invention without departing from the spirit orscope of the invention. Thus, it is intended that the present inventioncovers the modifications and variations of this invention provided theycome within the scope of the appended claims and their equivalents.

The present system and processes use detection of changes incharacteristics of reflected light (117) to overcome obscurationlimitations to allow sensing of non-direct line of sight regions. Thesystem uses active optical sensing in which an electrically andoptically controlled light source is pulsed to emit optical energy at aspecified intensity, duration, and pulse rate. Accordingly, for purposesherein, some embodiments of this disclosure refer to a “pulse of light”which may be considered a spatially and temporally modulated lightpattern emitted from a light source (110, 210), such as anelectro-optically controlled light source. The pulses of light result inreflected optical energy available for reception and processing at asensor or camera or any imaging device (110A, 110B, 210) having a fieldof view adjusted to receive the reflected light. Preferably an imagingdevice (110A, 110B, 210) used as a sensor for purposes herein utilizes apixelated sensing surface for receiving reflected light thereon. Forexample, the reflected light received at a sensor, camera, or any imagedetection apparatus may be a function of the shape and materialproperties of the surfaces from which the light is reflected. Thereflected optical energy is reflected off of objects (160, 320, 360,490, 495) (both living and inanimate) within a given three-dimensionalspace (190), and the system includes a multi-pixel detector whichdetects and quantifies the intensity of light at each pixel. Throughadditional signal processing steps, the detector and an associated imageprocessor (i.e., a computer processor connected to computerized memoryand the detector) can be used to form an intensity and/or distance imageas shown in FIG. 3. In this regard, the sensor may be an electro-opticalsensor configured to convert received/reflected light (117) into aplurality of electronic signals that can be processed to emulatepatterns of objects in a given field of view.

Furthermore, characteristics of motion of non-living objects (320, 490,495) differ greatly from motion associated with living creatures (160),and, therefore, are particularly distinguishable from animals whichexhibit somatic and autonomic movements. Accordingly, the systemsdescribed herein (for example, those implemented with electro-opticalsensors (110, 210) configured as 3-D Time of Flight sensors, referred toherein as “3-D TOF” sensors) can be used to discriminate and categorizevarious objects in a 3-D space, such as a vehicle interior or vehiclecabin (115), by analyzing how optical energy is affected or changed froma reference point when the optical energy is reflected off of thevarious objects. In one embodiment, the reference point may be acollective set of pixel data from an electro-optical sensor (110, 210),and the change may be measured as change in the time domain, frequencydomain, or spatial domain. The change may also be a change in a propertyof the optical signal, such as intensity. According to exampleembodiments discussed herein, this disclosure illustrates systems andmethods with which the objects can be classified within a vehiclecompartment.

For purposes herein, therefore, and without limiting the disclosure toany one embodiment, categories of objects in a vehicle may be classifiedas follows, using the systems and methods described herein, to furtherillustrate the embodiments of this disclosure:

Class (1) Living, dynamic (160) (e.g., adults, children, toddlers,infants, pets);

Class (2) Living, non-dynamic (e.g., plants);

Class (3) Non-living, dynamic (360) (e.g., motive tools (e.g., batteryoperated drills, shakers) or toys (battery operated, moving dolls, cars,action figures);

Class (4) Non-living, semi-static (140, 240, 340) (e.g., objects thathave a variety of controlled degrees of mechanical freedom or settings,for example, adjustable seating systems, seat belts, steering wheels,arm rests); and

Class (5) Non-living, static (219) (e.g., objects fixed relative to thesensor such as headliner, supports such as a/b pillars around windowsand structural details in a vehicle, floor/carpet/light mounts,seat-belt apertures, styling patterns (fabric patterns/trim).

In some applications, it may be desirable to sense objects of class (1)from all other objects (classes (2)-(5)). A 3-D time of flight (TOF)camera (110, 210) can be used in this way, but such a camera, or sensor,is limited to detecting objects that are fully, or at least mostly, inthe direct line of sight (117C, 117D) of the camera. For thisdisclosure, the inclusion of optically reflective features, such as atleast one reflective surface (219, 317) positioned in a vehiclecompartment (115), can greatly expand the overall effective field ofview of a camera/sensor within the vehicle compartment. For example, andwithout limiting this disclosure, within multiple classes, a system ofreflective surfaces and associated camera/sensor installations in avehicle can be used for at least the following purposes:

A. In regard to Class (5), a method and system as described herein mayprovide fixed calibration references for periodic or continuouscalibration of a detector and illuminator system (110, 210) (e.g., acamera or 3-D TOF sensor and an associated light source);

B. In regard to Class (4), a method and system as described herein mayserve to provide dynamic calibration references for assessing theadjustable state of a non-living, semi-static object (140, 240,340)(e.g. seat moved forward, seat moved back, belt position andorientation, and, for example, proper belt use based on detection anddiscrimination of a target pattern on the belt). Another example isapplying a reflective pattern surface (219) on a seat head-rest surfaceand being able to determine the seat angle based on a position andreflected light intensity pattern for any possible mechanical degree offreedom for the seat (240).

C. In further regard to Class (4), a method and system as describedherein may provide a means to reflect light into those areas within avehicle compartment obscured from direct line of site of a sensor orcamera. Light reflected back from objects within the obscured area (190)can be detected by a properly positioned sensor (110) and provideinformation about objects within the obscured area, for example patternrecognition and/or motion of objects within the obscured area as shownin FIGS. 3-4.

D. In regard to Class (1), a method and system as described herein maymonitor seatbelt (215) positions and/or an optically active/reflectivepattern (219) on or near the seat belt relative to at least oneoccupant. In this way, the method and system enable a procedure toestimate a stature of the occupant and/or whether a seatbelt (215) isout of position from a previous position, a desired position, or aposition that is required by transportation regulations. A reflectivesurface (130, 219, 317) where infrared (IR) light reflecting andinfrared (IR) light absorbing surfaces can be used together to achieveany desired pattern, enhance detectability through pixel intensityand/or detected distance from the sensor. A printed or embroideredinfrared (IR) sensitive/reflective pattern (219), which can include oneor more kinds of material, can be placed on the belt (215) as to beeasily detected/monitored by the active sensor to assess informationabout the occupant (e.g. the aforementioned proper belt use, occupantsize/stature).

According to principles of the present invention, at least one 3-D TOFsensor, such as but not limited to, those marketed by ESPROS PHOTONICS®AG, is placed at a respectively predetermined location within a vehiclecabin. For example, sensors such as those sold under ESPROS's EPC™series emit sinusoidal infrared light pulses with 5-20 MHz pulserepetition. While the present example contemplates sensors along thelines of the ESPROS EPC610™, EPC635™, and EPC660™, one of skill in theart would appreciate that other active electro-optical sensors can beused without deviating from the spirit and scope of this disclosure.Nothing in this disclosure should be interpreted as limiting thisdescription to any one kind or brand of sensor, camera, or imagedetection apparatus. References to particular devices and brand names ormerely for example. For example, although not described in detail here,there are increasingly wider bandwidth sensors (multi-spectral) thatcould be considered in the future, perhaps with wavelengths that aretunable or having several different active optical energy light sourcesto cover sensed bandwidth. These kinds of developments can be usedaccording to principles of the present disclosure. Light operatingfrequencies may vary depending on the sensor used. One example sensorutilizes a wavelength in the range from 300 nm to 1050 nm. Sub-ranges inthe near infrared spectrum of 950 nm-1050 nm are well within the scopeof this disclosure as well.

The reflective material (e.g., the pattern (219) on a seatbelt (215),specifically chosen for its operability with the selected 3-D TOFsensor, is applied to predetermined locations within the vehicle cabin(115). As illustrated in the figures, the reflective material maycomprise, in one non-limiting embodiment, a reflective surface (130,219, 317) having a pre-determined configuration with a pattern of highlyreflective regions and highly absorptive regions. In embodimentsutilizing a reflective surface having a pattern as noted, the patternmay be configured to accommodate sensor parameters such that lightreflected from the pattern to an associated sensor is readilydiscernible by sensor software. In this way, the sensor software moreefficiently discerns positions, directional changes, and the like for acomponent bearing the pattern. In certain embodiments, the reflectivesurface (130, 219, 317) may include different materials exhibitingrespective absorptive and reflective properties designed for aparticular installation. The reflective and absorptive regions may beconfigured to exhibit optimal characteristics at desired wavelengthsthat coincide with an associated optimal operating range for anassociated detector system. The reflective and absorptive regionsdescribed herein are only one implementation of a reflective surface,but it is noteworthy that non-limiting embodiments of the reflectivesurface may include a reflective, patterned surface configured fortracking by the sensor when light is reflected to the sensor from arecognizable pattern.

For example, a system and method disclosed herein may utilize anarrangement of multiple reflective surfaces (130 A-130D) and associatedcameras/sensors (110A, 110B) such that at least a primary portion of afirst reflective surface (130A, 130B, 130D) is placed within or mostlywithin a direct line of sight (117A, 117C, 117D) to the 3-D TOF sensor(110A, 110B). Additional reflective material, i.e., at least a secondreflective surface (130C), may be placed in a location not in a directline of sight of the sensor, but in a line of sight (117B) of a primaryportion of the first reflective surface (130A). Reflective materialbearing the reflective surfaces may be placed on a movable component(e.g., the above described semi-static structures (140, 240) such asvehicle seats, windows, arm rests, and the like) that are adjustable tomore than one physical position. Reflective surfaces on these movablebut semi-static components within a vehicle cabin may be initiallywithin the line of sight of the 3-D TOF sensor when the moveablecomponent is in one position, but may be out of line of sight of the 3-DTOF sensor if the component, or a portion of the component, is displacedfrom the one position. For example, placement of reflective surfacescould be configured on a seat belt (215) that is in an image sensor'sline of sight when properly worn by a vehicle occupant, but out of theline of sight if not in use by a vehicle occupant. This arrangement mayallow for the sensor and associated computerized components to providealarm functions when the seat belt is used improperly.

The reflective material, the reflective surfaces, and the configurationof the reflective and absorptive portions of respective patterns shouldbe chosen according to the sensor being used. Semi-reflective surfacesother than simple mirrors, are also covered in this disclosure. Thepattern used in a reflective surface can also be specified to optimizedetectability within an associated sensor's wavelength range, which may,in certain example circumstances, not be detected at human visiblewavelengths.

The surface does not need to be one hundred percent reflective oruniform, but in one non-limiting example the reflective surface is fixedrelative to a sensor assembly, such as an illuminator and associatedcamera. When a plurality of reflective surfaces are properly configuredwithin a three-dimensional space, such as a vehicle compartment, motionin a hidden space that is within a field of view of a reflective orsemi-reflective surface will be subject to sensing as discussed herein,resulting in a changing pattern as sensed by the TOF imager. In oneembodiment, the combination of the reflective surface and sensor needsufficient resolution to distinguish that something is moving, ratherthan necessarily distinguishing “what” is moving. In other words, thisdisclosure encompasses optically sensitive sensors and cameras for whichresolution of the sensed optical energy or image may be tailored tomatch the need at hand. Recognizing movement of an object within an arearequires significantly lower resolution compared to identifying theexact object therein. Other embodiments may sense both the movement andan identification of the object that is moving.

FIG. 1 illustrates a vehicle cabin (115) with exemplary interior vehiclecomponents, in this case, passenger seats. As illustrated in thisexample, an active optical sensor (110B) is placed in a front-wardlocation of the vehicle cabin (115) and in a rear-ward location (110A)in the vehicle cabin. Highly reflective surfaces (130) are created,e.g., by application of a reflective material, at predeterminedlocations within the vehicle cabin. Locations of the reflective surfacesare chosen to maximize the intensity of primary, secondary, tertiary,etc., reflections from an area of interest (190) in the cabin to atleast one of the active optical sensors. Placement of the reflectivesurfaces (130) can be used to allow measurement of reflection ofobscured areas within the vehicle cabin.

FIG. 2 illustrates reflective surfaces on vehicle components, namelyseat belts (215). As illustrated, when a seat belt is worn by a vehicleoccupant, the reflective material (219) is in line of sight of a frontmounted TOF sensor (not shown). Serial pulses from the TOF sensor canprovide information as to whether the emitted signal is reflected backto the camera or not, indicating whether the reflective material on aparticular seat belt within the cabin is in a wearing position. Thisinformation can alone be used to determine whether a passenger occupiesa seat within the cabin, or can be compared with other vehicle occupantsensor mechanisms to indicate whether a passenger may be in the vehiclebut not wearing a seat belt. The present system can also be used todetermine location (by measurement of return pulse timing) to determinethe location of a passenger or component within the vehicle cabin. Thereflective surface/material may have a predetermined pattern to assistin the location/distance measurements. In addition, certain highlyreflective patterned surfaces, such as those surfaces encompassingmirrors, may be added to the vehicle cabin to aid in reflection.

According to principles of the present invention, in one exampleembodiment, light may be directed from a light source in a vehiclecompartment (e.g., the 3-D TOF sensor may incorporate its own lightsource/illumination projector). The preferred optical signal directed toan object and reflected back to the sensor may be “coded” in the time orfrequency domain or even in regard to physical dimensions, or spatialdomain, for the optical signal. Accordingly, the relevant sensor onlymeasures/responds to a predetermined coded light, while not respondingto un-coded, external or ambient light and lighting changes. Moreover,current vehicle windows are coated or otherwise designed to filtercertain light wavelengths. Thus, the TOF sensor operating frequency canbe selected to be one that is filtered by the windows such that externallight (e.g., light noise) within the vehicle cabin is minimized in theoperating wavelength range for an associated sensor and system describedherein.

In an aspect of the present invention, a 3-D TOF sensor is placed at anuppermost, farthest forward position of the vehicle cabin, for example,on the headliner by the rearview mirror or integrated with the rearviewmirror assembly, with primary viewing angle facing toward the vehiclerear. Another 3-D TOF sensor is placed at the uppermost, farthestrearward position of the vehicle cabin, for example, by the upper rearutility light in a mini-van or SUV, with primary viewing angle facingtoward the vehicle front. The viewing angle of the lens of the 3-D TOFsensor is contemplated to be approximately 90° to approximately 120°with a standard lens, but may be adapted by changing the lens asnecessary for various applications.

FIG. 3 shows both an exemplary depth image produced by the 3-D TOFsensor and an exemplary intensity image of a footwell in a vehiclemonitored through a fixed mirror. Illustrated are the mirror (317), amoving doll (360) as a model and a blanket (320). FIG. 4 shows intensityand depth images of the footwell at time T1 and time T2. A closeinspection of the intensity images at T1 and T2 shows a slight change inthe image reflected in the mirror (405), indicating movement of theobject (the moving doll) (360) reflected in the mirror (405) betweentime T1 and time T2. The depth image at time T2 shows that there hasbeen a depth change between time T1 and time T2, characterized by achange in depth measured by the 3-D TOF sensor. The depth changes areindicated by an overlay in the figure of boxes that illustrate where thedepth changes have been sensed. The stripes indicate unchanged depth ata particular location.

This technique provides a unique, low cost method to improve theperformance and extend applications of active optical sensing systemthrough calibration references (e.g. look for known/fixed locationswhere an object “should” be). Semi-dynamic state information can bedetermined through detection of references. Information about obscuredregions (190) can be obtained from a single sensor. In otherembodiments, output from additional sensors other than 3D TOF sensorsmay be used by associated software to add more opportunities forartificial intelligence operations. For example, with a moving object,image processing (standalone or with support from available sensors suchas accelerometer signals) can help separate if a sensed motion isinduced by the movement of the vehicle or the living object picked upwithin the field of view of the sensor. Similarly, comparing series ofthe sensed images along with known conditions of the vehicle movement,direction, velocity, acceleration and the like, can be used to determineor verify classifications of objects as living or dynamic in some way.

In one embodiment, a system for classifying objects within an identifiedvolume of space (190) includes an electro-optical sensor (110) incommunication with a processor and computerized memory storingcomputer-implemented software thereon. A reflective surface (130)capable of reflecting an optical signal back to the electro-opticalsensor is positioned proximate the identified volume of space, whereinthe reflective surface is at least partially within a first line ofsight (117A, 117C, 117D) originating from the electro-optical sensor.The reflective surface is positioned relative to the identified volumeof space such that a second line of sight originating from thereflective surface encompasses at least a portion of the identifiedvolume of space. The optical signal reflected from the reflectivesurface back to the electro-optical sensor includes image data gatheredby the reflective surface via the second line of sight (117B) into theidentified volume (190). If characteristics of the optical signal asreflected by the reflective surface at a first time differ from latercharacteristics of the optical signal as reflected by the reflectivesurface at a second time, the sensor identifies movement datacorresponding to the objects within the identified volume via thesoftware. A first line of sight includes the image data gathered by thereflective surface such that an effective field of view for theelectro-optical sensor comprises the first line of sight and at least aportion of the second line of sight. The system further includescomputer controlled alarm functions stored in the memory and incommunication with the processor, wherein the alarm functions areconfigured to activate an alarm in accordance with predetermined rulesestablished in the software for categories of objects identified by thesoftware with the movement data.

In another embodiment, the system further includes computer controlledalarm functions stored in the memory and in communication with theprocessor, wherein the alarm functions are configured to activate analarm in accordance with predetermined rules established in the softwarefor categories of objects identified by the software with the movementdata.

In another embodiment, an indirect reflective surface (130C) defining athird line of sight (117F), wherein the indirect reflective surface iswithin the second line of sight (117B) originating at the reflectivesurface (130A) but is not within the first line of sight (117A)originating from the electro-optical sensor (110), and wherein the thirdline of sight originating (117F) from the indirect reflective surfaceincludes at least a portion of the identified volume (190) that is notin the first line of sight originating with the active electro-opticalsensor.

In another embodiment, a system for detecting proper positioning of adevice includes the above noted electro-optical three-dimensional (3D)image sensor (110) and a reflective surface (130) capable of reflectinga wavelength corresponding to a wavelength operating range for theelectro-optical three-dimensional (3D) image sensor, wherein thereflective surface is on a component such that, if the component is in aprimary position. The reflective surface reflects first image data intoa first line of sight (117A, 117C, 117C) originating at theelectro-optical three-dimensional (3D) image sensor, and wherein thesensor is configured to generate movement data regarding the componentfrom differences in the characteristics of light reflected by thereflective surface to the electro-optical three-dimensional (3D) imagesensor. The component may be a semi-static structure (215, 240) within avehicle, and the reflective surface reflects image data into the firstline of sight for reception by the electro-optical (3D) sensor, whereinthe software accesses position data corresponding to allowed degrees offreedom within which the component (215, 240) is allowed to operatewithout triggering the alarm, and wherein the software utilizes theposition data and the image data in a decision to trigger the alarm.

An associated method allows for detecting an object within an enclosedspace (115), and the steps include transmitting a pulse of light from athree-dimensional (3D) time of flight camera (110) into the enclosedspace, detecting presence or absence of a returned pulse of lightreflected to the three-dimensional (3D) time of flight camera; andmeasuring changes in characteristics of the returned pulse of light todetermine changes in objects within the enclosed space. Presence of thereturned pulse of light indicates a component (215, 240) within theenclosed space is in a primary position, and absence of the returnedpulse indicates displacement of the component. If characteristics oflight returned to the three-dimensional (3D) time of flight camera at afirst time differ from characteristics of light returned to thethree-dimensional (3D) time of flight camera at a second time, thesensor indicates that either a semi-static or a dynamic object ispresent in the enclosed space.

This disclosure includes concepts for developing software stored onnon-volatile computer readable media to connect to, interact with,and/or control the imaging and sensing system described herein. Forexample, a processor and associated memory may be configured to executecomputer readable commands in vehicle systems if a 3D TOF sensor detectsmovements in the image when the car is not in motion, a factor thatwould help verify a living, dynamic status for that object. Similarly,and without limiting this disclosure to any one embodiment, a detectedmovement may be identified as actual shifting of a non-living,non-dynamic class object (due to vehicle movement and braking). A seriesof images, analyzed by software executed by the processor herein, coulddistinguish that object from a living, dynamic object (possibly by thepatterns of motion not matching the vehicle motion).

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device. A computer readable signal medium may include apropagated data signal with computer readable program code embodiedtherein, for example, in baseband or as part of a carrier wave. Such apropagated signal may take any of a variety of forms, including, but notlimited to, electro-magnetic, optical, or any suitable combinationthereof. A computer readable signal medium may be any computer readablemedium that is not a computer readable storage medium and that cancommunicate, propagate, or transport a program for use by or inconnection with an instruction execution system, apparatus, or device.Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing. Computer program code for carrying out operations foraspects of the present invention may be written in any combination ofone or more programming languages, including an object orientedprogramming language such as Java, Smalltalk, C++ or the like andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The program codemay execute entirely on the user's computer, partly on the user'scomputer, as a stand-alone software package, partly on the user'scomputer and partly on a remote computer or entirely on the remotecomputer or server. In the latter scenario, the remote computer may beconnected to the user's computer through any type of network, includinga local area network (LAN) or a wide area network (WAN), or theconnection may be made to an external computer (for example, through theInternet using an Internet Service Provider).

Aspects of the present invention are described below with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to implementations ofthe invention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks. These computerprogram instructions may also be stored in a computer readable mediumthat can direct a computer, other programmable data processingapparatus, or other devices to function in a particular manner, suchthat the instructions stored in the computer readable medium produce anarticle of manufacture including instructions which implement thefunction/act specified in the flowchart and/or block diagram block orblocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The terminology used herein is for the purpose of describing particularimplementations only and is not intended to be limiting of theinvention. As used herein, the singular forms “a”, “an” and “the” areintended to include the plural forms as well, unless the context clearlyindicates otherwise. It will be further understood that the terms“comprises” and/or “comprising,” when used in this specification,specify the presence of stated features, integers, steps, operations,elements, and/or components, but do not preclude the presence oraddition of one or more other features, integers, steps, operations,elements, components, and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thedescription of the present invention has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to the invention in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the artwithout departing from the scope and spirit of the invention. Theimplementation was chosen and described in order to best explain theprinciples of the invention and the practical application, and to enableothers of ordinary skill in the art to understand the invention forvarious implementations with various modifications as are suited to theparticular use contemplated.

While various embodiments of the present invention have been describedabove, it should be understood that they have been presented by way ofexample only, and not limitation. It will be apparent to persons skilledin the relevant art that various changes in form and detail can be madetherein without departing from the spirit and scope of the presentinvention. Thus, the breadth and scope of the present invention shouldnot be limited by any of the above-described exemplary embodiments, butshould be defined only in accordance with the following claims and theirequivalents.

What is claimed is:
 1. A system for classifying objects within a space,comprising: an electro-optical sensor in communication with a processorand computerized memory storing computer-implemented software thereon; asurface of non-uniform reflectivity reflecting a plurality of opticalsignals back to the electro-optical sensor over a time period, whereinthe optical signals comprise image data that is non-uniform in thespatial domain over the time period; wherein the surface of non-uniformreflectivity is entirely within a first line of sight originating fromthe electro-optical sensor; wherein the surface is positioned relativeto an identified volume of space such that a second line of sightoriginating from the surface encompasses at least a portion of theidentified volume of space; wherein the optical signals reflected fromthe surface back to the electro-optical sensor comprise image datatransmitted by an entirety of the surface via the second line of sightinto the identified volume; wherein the electro-optical sensoridentifies movement data corresponding to the objects within theidentified volume on a pixel by pixel basis via the software.
 2. Asystem according to claim 1, wherein the electro-optical sensorcomprises an array of optically sensitive pixels receiving the opticalsignal and transmitting respective image data signals to the processorfor analysis by the software.
 3. A system according to claim 1, whereinthe software is configured to use the movement data to categorize theobject as static, semi-static, or dynamic.
 4. A system according toclaim 1, wherein the first line of sight includes the image datagathered by the surface such that an effective field of view for theelectro-optical sensor comprises the first line of sight and at least aportion of the second line of sight.
 5. A system according to claim 1,further comprising computer-controlled alarm functions stored in thememory and in communication with the processor, wherein the alarmfunctions are configured to activate an alarm in accordance withpredetermined rules established in the software for categories ofobjects identified by the software with the movement data.
 6. The systemof claim 1, further comprising an indirect reflective surface defining athird line of sight, wherein the indirect reflective surface is withinthe second line of sight originating at the surface of non-uniformreflectivity but is not within the first line of sight originating fromthe electro-optical sensor, and wherein the third line of sightoriginating from the indirect reflective surface includes at least aportion of the identified volume that is not in the first line of sightoriginating with the active electro-optical sensor.
 7. The system ofclaim 6, wherein the identified volume is not within the first line ofsight or the second line of sight.
 8. A system for detecting properpositioning of a movable, component in a vehicle cabin comprising: anelectro-optical three-dimensional (3D) image sensor; and a surface ofnon-uniform reflectivity attached to the movable component andreflecting a wavelength of light corresponding to a wavelength operatingrange for the electro-optical three-dimensional (3D) image sensor,wherein the surface of non-uniform reflectivity is attached to themoveable component such that, if the moveable component is in a primaryposition, the surface reflects first image data into a first line ofsight originating at the electro-optical three-dimensional (3D) imagesensor; and wherein the sensor is configured to generate movement dataon a pixel by pixel basis and from spatially non-uniform imagescomprising pixel data representing at least a portion of the componentand the surface of non-uniform reflectivity; wherein the sensorgenerates the movement data from differences in the characteristics ofreflected light from the surface of non-uniform reflectivity to theelectro-optical three-dimensional (3D) image sensor, wherein themovement data is of a sufficient resolution to identify the differenceson a pixel by pixel basis.
 9. A system according to claim 8, furthercomprising a processor and computerized memory storingcomputer-implemented software that is in data communication with theelectro-optical three-dimensional (3D) sensor, wherein the softwareutilizes the movement data retrieved from the sensor during a timeperiod to trigger at least one alarm regarding the primary position ofthe component.
 10. A system according to claim 9, wherein the componentis a semi-static structure within a vehicle, and the surface reflectsimage data into the first line of sight for reception by theelectro-optical (3D) sensor, wherein the software accesses position datacorresponding to allowed degrees of freedom within which the componentis allowed to operate without triggering the alarm, and wherein thesoftware utilizes the position data and the image data in a decision totrigger the alarm.
 11. A system according to claim 10, wherein if thecomponent is in a different position, the surface reflects second imagedata into the first line of sight, and differences between the firstimage data and the second image data are used by the sensor to calculatethe movement data.
 12. The system of claim 8, wherein the wavelength oflight is in the visible spectrum.
 13. A method of detecting an objectwithin an enclosed space with a three-dimensional (3D) time of flightcamera, comprising: using a surface of non-uniform reflectivity,reflecting a plurality of optical signals back to the 3D time of flightcamera over a time period, wherein the optical signals comprise imagedata that is non-uniform in the spatial domain over the time period;wherein the surface of non-uniform reflectivity is entirely within afirst line of sight originating from the 3D time of flight camera;transmitting a coded pulse of light from the three-dimensional (3D) timeof flight camera into the enclosed space; detecting a code from thecoded pulse of light; after detecting the code, detecting presence orabsence of a returned pulse of light reflected to the three-dimensional(3D) time of flight camera; and measuring changes in characteristics ofthe returned pulse of light to determine changes in objects within theenclosed space.
 14. The method of claim 13, wherein presence of thereturned pulse of light indicates a component within the enclosed spaceis in a primary position, and absence of the returned pulse indicatesdisplacement of the component.
 15. The method of claim 13, wherein ifcharacteristics of light returned to the three-dimensional (3D) time offlight camera at a first time differ from characteristics of lightreturned to the three-dimensional (3D) time of flight camera at a secondtime, the sensor indicates that either a semi-static or a dynamic objectis present in the enclosed space.
 16. The method of claim 13, wherein atleast one surface reflects image data from the objects to the threedimensional (3D) time of flight camera, and the image data characterizeseither a depth of at least one of the objects or an intensity of anoptical signal reflected from the surface.
 17. The method of claim 16,wherein the object is a vehicle component, and further comprisingtriggering an alarm within a vehicle based on either the depth or theintensity.
 18. The method of claim 13, wherein each of the objectscomprises a respective surface that reflects the pulse of light from afirst portion of the surface and absorbs the light into a second portionof the surface.
 19. The method of claim 13, further comprising tailoringthe resolution of the three dimensional (3D) time of flight camera todetect movement of the objects within an identified volume of space. 20.The method of claim 13, wherein the coded pulse of light is coded in atleast one of the time domain, the frequency domain, and the spatialdomain.